We present the challenging task of automatically creating a high-level Wikipedia-style article that aggregates information from multiple diverse videos about real-world events, such as natural disasters or political elections. Videos are intuitive sources for retrieval-augmented generation (RAG), but most contemporary RAG workflows focus heavily on text and existing methods for video-based summarization focus on low-level scene understanding rather than high-level event semantics. To close this gap, we introduce WikiVideo, a benchmark consisting of expert-written articles and densely annotated videos that provide evidence for articles' claims, facilitating the integration of video into RAG pipelines and enabling the creation of in-depth content that is grounded in multimodal sources. We further propose Collaborative Article Generation (CAG), a novel interactive method for article creation from multiple videos. CAG leverages an iterative interaction between an r1-style reasoning model and a VideoLLM to draw higher level inferences about the target event than is possible with VideoLLMs alone, which fixate on low-level visual features. We benchmark state-of-the-art VideoLLMs and CAG in both oracle retrieval and RAG settings and find that CAG consistently outperforms alternative methods, while suggesting intriguing avenues for future work.

Retrieval and recommendation are two essential tasks in modern search tools. This paper introduces a novel retrieval-reranking framework leveraging Large Language Models (LLMs) to enhance the spatiotemporal and semantic associated mining and recommendation of relevant unusual climate and environmental events described in news articles and web posts. This framework uses advanced natural language processing techniques to address the limitations of traditional manual curation methods in terms of high labor cost and lack of scalability. Specifically, we explore an optimized solution to employ cutting-edge embedding models for semantically analyzing spatiotemporal events (news) and propose a Geo-Time Re-ranking (GT-R) strategy that integrates multi-faceted criteria including spatial proximity, temporal association, semantic similarity, and category-instructed similarity to rank and identify similar spatiotemporal events. We apply the proposed framework to a dataset of four thousand Local Environmental Observer (LEO) Network events, achieving top performance in recommending similar events among multiple cutting-edge dense retrieval models. The search and recommendation pipeline can be applied to a wide range of similar data search tasks dealing with geospatial and temporal data. We hope that by linking relevant events, we can better aid the general public to gain an enhanced understanding of climate change and its impact on different communities.

Object Re-Identification (Re-ID) aims to identify and retrieve specific objects from varying viewpoints. For a prolonged period, this field has been predominantly driven by deep convolutional neural networks. In recent years, the Transformer has witnessed remarkable advancements in computer vision, prompting an increasing body of research to delve into the application of Transformer in Re-ID. This paper provides a comprehensive review and in-depth analysis of the Transformer-based Re-ID. In categorizing existing works into Image/Video-Based Re-ID, Re-ID with limited data/annotations, Cross-Modal Re-ID, and Special Re-ID Scenarios, we thoroughly elucidate the advantages demonstrated by the Transformer in addressing a multitude of challenges across these domains. Considering the trending unsupervised Re-ID, we propose a new Transformer baseline, UntransReID, achieving state-of-the-art performance on both single-/cross modal tasks. Besides, this survey also covers a wide range of Re-ID research objects, including progress in animal Re-ID. Given the diversity of species in animal Re-ID, we devise a standardized experimental benchmark and conduct extensive experiments to explore the applicability of Transformer for this task to facilitate future research. Finally, we discuss some important yet under-investigated open issues in the big foundation model era, we believe it will serve as a new handbook for researchers in this field.

I recently needed to learn text mining for a project at work. I generally learn more quickly with a real-world project. So, I turned to a topic I love: Wilderness, to see how I could apply the skills of text scrubbing and natural language processing. You can clone my Git repo for the project or follow along in the post below. The first portion of this post will cover web scraping, then text mining, and finally analysis and visualization.

AI is being applied to whale research, especially to understand what whales are trying to communicate in the audible sounds they make to each other in the ocean. For example, marine biologist Shane Gero has worked to match clicks coming from whales around the Caribbean island nation of Dominica, to behavior he hopes will reveal the meanings of the sounds they make. Gero is a behavioral ecologist affiliated with the Marine Bioacoustics Lab at Aarhus University in Denmark, and the Department of Biology of Dalhousie University of Halifax, Nova Scotia. Gero works with a team from Project CETI, a nonprofit that aims to apply advanced machine learning and state-of-the-art robotics to listen to and translate the communication of whales. Project CETI has recently announced a five-year effort to build on Gero's work with a research project to try to decipher what sperm whales are saying to each other, according to a recent account in National Geographic.

Machine learning (ML) is how a system learns and adapts its processes from the patterns found in large amounts of data. When we think of machine learning, some prominent examples come to mind. For instance, the way product recommendations on Amazon are eerily similar to Google searches you've done. The scope of machine learning extends far beyond what we know of and see in our daily lives. Since machine learning is a relatively new field, the limits of its application are constantly pushed outward.

Machine Learning refers to the process through which a computer learns and changes its operations based on patterns identified in vast quantities of data. When we think about machine learning, we think of a few well-known instances. For example, the way Amazon recommends products is remarkably similar to Google searches you've done. Machine learning's reach is far broader than what we are familiar with and observes in our daily lives. Because machine learning is such a young science, the boundaries of its applicability are continuously being pushed outside. Virtual personal assistants were once the stuff of fantasies, but now they can be found in every other home.

Researchers in Norway are testing how salmon in a commercial fish farm might react to being regularly monitored by an underwater robots. While fish farms are typically uneventful environments, they still require oversight to ensure the captive fish are safe and healthy, a task most commercial fish farms assign to a human diver. Maarja Kruusmaa and a team of researchers at the Norwegian University of Science and Technology wanted to test how fish would respond to being watched over by robots instead of people. 'The happier the fish are, the healthier the fish are, the better they eat, the better they grow, the less parasites they have and the less they get sick,' Kruusmaa told New Scientist. The team used two different underwater robots to test whether the fish would react differently based on the size and propulsion method.

National Oceanic and Atmospheric Administration scientists are preparing to use machine learning (ML) to more easily monitor threatened ice seal populations in Alaska between April and May. Ice flows are critical to seal life cycles but are melting due to climate change -- which has hit the Arctic and sub-Arctic regions hardest. So scientists are trying to track species' population distributions. But surveying millions of aerial photographs of sea ice a year for ice seals takes months. And the data is outdated by the time statisticians analyze it and share it with the NOAA assistant regional administrator for protected resources in Juneau, according to a Microsoft blog post.

Moreland's project combines AI technology with improved cameras on a NOAA turboprop airplane that will fly over the Beaufort Sea north of Alaska this April and May, scanning and classifying the imagery to produce a population count of ice seals and polar bears that will be ready in hours instead of months. Her colleague Manuel Castellote, a NOAA affiliate scientist, will apply a similar algorithm to the recordings he'll pick up from equipment scattered across the bottom of Alaska's Cook Inlet, helping him quickly decipher how the shrinking population of endangered belugas spent its winter.